Mac Studio M4 Max - Secure AI Gateway

Production Architecture Secure HTTPS Transit

API Consumer / Clients

Terminal, IDE Extension, App

↓ [HTTPS: 443]

Port 443

Nginx Reverse Proxy

SSL Termination | Host: llm.i2.services

↓ [Local Loopback]

Port 4000

LiteLLM Proxy Router

Auth (Bearer Token) | PG Backend | Drop Params

↓

Port 8081

MLX Engine (`gemma-4-31b-8bit`)

Model: Gemma 4 31B (8-bit)

↓

Port 8080

MLX Engine (`gemma-4-12B-8bit`)

Model: Gemma 4 12B (8-bit)

Internal clients target https://llm.i2.services/v1/chat/completions. The proxy terminates SSL securely and forwards queries locally.

Secure Properties

Host Domain llm.i2.services

SSL Provider Internal CA Signed

Host Processor Apple M4 Max

Active Backends Dual Gemma 4 (MLX)

Nginx Memory Use < 15 MB

Max Body Limit 50 MB (For Large Context)

Stream Buffering Disabled (Off)

Operational Playbook & Configs

server {
    listen 443 ssl;
    server_name llm.i2.services;

    ssl_certificate     /opt/homebrew/etc/nginx/ssl/gateway.crt;
    ssl_certificate_key /opt/homebrew/etc/nginx/ssl/gateway.key;
    
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;

    client_max_body_size 50M;

    location / {
        proxy_pass http://127.0.0.1:4000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        
        # Vital configurations for LLM token streaming
        proxy_buffering off;
        proxy_read_timeout 300s;
        proxy_connect_timeout 300s;
    }
}

server {
    listen 80;
    server_name llm.i2.services;
    return 301 https://$host$request_uri;
}

# 1. Establish Nginx SSL staging directories on Mac Studio
mkdir -p /opt/homebrew/etc/nginx/ssl
cd /opt/homebrew/etc/nginx/ssl

# 2. Generate Private Key (gateway.key) and Certificate Request (gateway.csr)
# Signed for domain name: llm.i2.services
openssl req -new -newkey rsa:2048 -nodes \
  -keyout gateway.key -out gateway.csr \
  -subj "/C=US/ST=State/L=City/O=Secunetics/CN=llm.i2.services"

# [Alternative] Generate with Subject Alternative Names (SAN) mapping both DNS and local Host IP
openssl req -new -newkey rsa:2048 -nodes \
  -keyout gateway.key -out gateway.csr \
  -subj "/C=US/ST=State/L=City/O=Secunetics/CN=llm.i2.services" \
  -addext "subjectAltName = DNS:llm.i2.services, IP:10.11.11.200"

# 3. Submit 'gateway.csr' to your Internal CA (Secunetics Root Certificate Authority).
# Once signed, copy the resulting public certificate file into:
# /opt/homebrew/etc/nginx/ssl/gateway.crt

# ~/local-ai/configs/litellm_config.yaml
model_list:
  - model_name: gemma-4-31b-8bit
    litellm_params:
      model: openai/mlx-community/gemma-4-31b-it-8bit
      api_base: http://127.0.0.1:8081/v1
      api_key: "not-needed"
      tpm: 200000
      rpm: 2000

  - model_name: gemma-4-12B-8bit
    litellm_params:
      model: openai/mlx-community/gemma-4-12B-it-8bit
      api_base: http://127.0.0.1:8080/v1
      api_key: "not-needed"
      tpm: 200000
      rpm: 2000

litellm_settings:
  drop_params: true
  set_verbose: false

environment_variables:
  MAX_STRING_LENGTH_PROMPT_IN_DB: 32000

general_settings:
  database_url: "postgresql://secadmin@localhost:5432/litellm_db"
  master_key: "sk_live_mac_studio_master_init_key_2026"
  store_model_in_db: true
  store_prompts_in_spend_logs: true

#!/bin/bash
# Local AI Engines TMUX Orchestration Script

# 1. Start GGUF/MLX Engine 1 - Gemma 4 31B (Port 8081)
tmux new-session -d -s engine-mlx-31b 'source ~/local-ai/venv-mlx/bin/activate && \
  mlx_lm.server \
  --model mlx-community/gemma-4-31b-it-8bit \
  --port 8081 --host 127.0.0.1'

# 2. Start MLX Engine 2 - Gemma 4 12B Base (Port 8080)
# Note: Since Gemma 4 12B is a multimodal (Unified) model, it must be run via mlx-vlm
tmux new-session -d -s engine-mlx-12b 'source ~/local-ai/venv-mlx/bin/activate && \
  mlx_vlm.server \
  --model mlx-community/gemma-4-12B-it-8bit \
  --port 8080 --host 127.0.0.1'

# 3. Start LiteLLM Gateway Router (Port 4000 bound to loopback and with truncation guardrail)
tmux new-session -d -s gateway-proxy 'export NO_DOCS=true && \
  export NO_REDOC=true && \
  export NO_OPENAPI=true && \
  export MAX_STRING_LENGTH_PROMPT_IN_DB=32000 && \
  source ~/local-ai/venv-mlx/bin/activate && \
  litellm --config ~/local-ai/configs/litellm_config.yaml --port 4000 --host 127.0.0.1'

echo "All engines dispatched in tmux background sessions: [engine-mlx-31b, engine-mlx-12b, gateway-proxy]"
tmux ls

curl -X POST https://llm.i2.services/v1/chat/completions \
  -H "Authorization: Bearer sk_live_mac_studio_master_init_key_2026" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemma-4-31b-8bit",
    "messages": [
      {
        "role": "user",
        "content": "Hello! Confirm you are running Gemma 4 31B over HTTPS."
      }
    ],
    "max_tokens": 4096
  }'

llm.i2.services — Secure Gateway

HTTPS Reverse Proxy Engaged